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Abstract 

In this paper, we investigate the problem of assignment of K identical servers to a set of N parallel 
queues in a time slotted queueing system. The connectivity of each queue to each server is randomly 
changing with time; each server can serve at most one queue and each queue can be served by at 
most one server per time slot. Such multi-server queueing systems were widely applied in modeling 
the scheduling (or resource allocation) problem in wireless networks. It has been previously proven 
that Maximum Weighted Matching (MWM) is a throughput optimal server assignment policy for such 
queueing systems HI, Q. In this paper, first we prove that for a symmetric system with i.i.d. Bernoulli 
packet arrivals and connectivities, MWM minimizes, in stochastic ordering sense, a broad range of cost 
functions of the queue lengths including total queue occupancy (or equivalently minimizes the average 
queueing delay). Then, we extend the model by considering imperfect services where it is assumed 
that the service of a scheduled packet fails randomly with a certain probability. We prove that the 
same policy is still optimal for the extended model. We also show that the results are still valid for 
some extended connectivity and arrival processes (e.g., Poisson arrival process). Second, for the multi- 
server queueing system with general stationary connectivity and arrival processes, we derive a linear 
algebraic characterization of the network stability region polytope which is needed to solve general 
utility optimization problems in such queueing systems. 

I. Introduction 

Optimal stochastic control of emerging wireless networks is one of the primary objectives 
in the design of such networks. In general, the main goal in the stochastic control of wireless 
networks is to distribute the shared resources in physical (e.g., power) and MAC layers (e.g., radio 
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interfaces, relay stations and orthogonal sub-channels) among multiple users such that certain 
stochastic performance attributes are optimized. While various performance attributes including 
the stable throughput region, power consumption and utility functions of the admitted traffic 
rates have been studied in many papers IfTI- lfTTl , average queueing delay has been considered 
far less in literature. This is due to the inherent difficulty of delay optimal scheduling problems 
in queueing systems with time varying channel conditions. In this paper, we consider a discrete 
time queueing system which is suitable in modeling of orthogonal resource allocation (e.g., radio 
interfaces, relay stations and communication channels) in multi-user wireless access networks. In 
our system, we model the available shared resources by a set of identical servers. The model also 
consists of a set of queues whose connectivities to each server are changing by time randomly. 
Therefore, resource assignment problem is equivalent to finding a matching between the queues 
and the servers at each time slot such that some performance objectives are optimized. It has been 
previously shown that Maximum Weighed Matching (MWM) is throughput optimal for such a 
system, i.e., it maximizes the stable throughput regiorl] [[Tl, [|2l. The MWM server assignment 
policy has also been extensively used in literature for treating the scheduling problem in crossbar 
packet switches [|5l, [El-Bll. In this paper, we first prove that for a symmetric system with i.i.d. 
Bernoulli packet arrivals and connectivities, MWM not only is throughput optimal, but also it is 
optimal in minimizing, in stochastic ordering sensed, a broad range of cost functions of queue 
lengths including total queue occupancy (or equivalently average queueing delay). We further 
characterize the stability region by a set of linear inequalities and obtain an upper bound for the 
average queueing delay. Such a linear algebraic characterization of stability region is useful in 
solving utility and fairness optimization problems in these queueing systems. 

The rest of this paper is organized as follows. In section |lll we will review the related work 
in this area of research and summarize the contributions we made in this paper Section UlI] 
describes the queueing model and the notation required throughout the paper In section |lVl 
we introduce Maximum Weighted Matching (MWM) policy whose delay optimality for the 

'stability region of a communication network is the closure of the set of all arrival rate vectors for which there exists a 
resource allocation policy that can stabilize the network. 

^We order two discrete time random processes A — {A{t)}^i and B = {B{t)}'^i stochastically as follows: We say A is 
stochastically less than B and we write A <st B if Fr{A{t) > r) < Pr(B(t) > r) for all t = 1,2, ... and all r £ R. The 
notion and relevant properties will be discussed in more detail in Section fV-AI 
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symmetric case of the introduced queueing model is proven in section |Vl In section |Vl we 
present one of the main contributions of this paper, that is proving delay optimality of MWM 
server assignment policy. In section |Vll we will discuss about the network stability region of the 
introduced queueing model and show that the stability region is a polytope. Then, we derive a 
linear algebraic characterization of the stability region polytope. In section IVIIl we present the 
simulation results where we compare the performance of MWM policy with the performance 
of two other server assignment policies in terms of average total queue occupancy. Section IVIIII 
summarizes the conclusions of the paper. 

II. Related Work and Our Contributions 

The problem of optimal server allocation in queueing systems with random connectivities was 
mainly addressed in [H, S, |[IOl-[[l3l, [[I8l-[l20l. In the authors introduced the notion of 
stability region of a general queueing network with time varying connectivities and they pro- 
posed Back-pressure algorithm as a throughput optimal resource allocation policy for queueing 
networks. In [3J, they considered a multi-queue single-server queueing system with random 
connectivities. They characterized the stability region by a set of linear inequalities and also 
proved that for a symmetric system (with the same arrival and connectivity parameters for all 
the queues), LCQ (Longest Connected Queue) provides the optimal performance in terms of 
average queue occupancy. 

In [[T2|. Maximum Weight (MW) policy was proposed as a throughput optimal server allocation 
policy for a multi-queue multi-server queueing system with stationary channel processes. In [[T3||. 
the authors characterized the network stability region of multi-queue multi-server systems with 
time varying connectivities. They also obtained an upper bound for the average queueing delay 
of AS/LCQ policy which is a throughput optimal server allocation policy for such systems. The 
results were further extended in [ |14| for more general stationary channel distributions (and not 
just i.i.d. Bernoulli channels). 

The authors in ifTSl considered a queueing model with a set of symmetrical parallel queues 
competing to attract service from K identical servers. The connectivity of each queue to all the 
servers is assumed to be the same at each time slot and during each time slot, each queue can 
attract at most one server. The authors proposed LCQ policy in which the servers are allocated 
to the K longest connected queues at each time slot. Using dynamic coupling and stochastic 
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ordering they proved the optimality of LCQ policy. 

The work in [fTO| . [[TT|. [fT9l . [[20l | focuses on the optimal server allocation problem in multi- 
queue multi-server queueing systems in terms of average queueing delay. In ifTOl . ifTTI . [[T9ll . 
the authors introduced MTLB (Maximum- Throughput Load-Balancing) policy and showed that 
this policy minimizes a class of cost functions including total average delay for the case of two 
symmetric queues. The work in [flOll considers this problem for general number of symmetric 
queues and servers. In [|20ll . a class of Most Balancing (MB) policies was characterized among 
all work conserving policies which are minimizing, in stochastic ordering sense, a class of cost 
functions including total queue occupancy. Note that in the model studied in ifTOll . [fTTIl . [fT3l . [[T9ll , 
II20I . there is no restriction on the number of servers that are serving a queue at each time slot. 
In [[T]|, it was shown that for a multi-queue multi-server system in which queues are restricted to 
attract at most one server at each time slot, maximum weighted matching is throughput optimal. 
The authors also considered the effect of infrequent channel state measurements on the network 
stability region. 

Our contributions in this paper are summarized as follows: 

• For a multi-server queueing system where each user is allowed to attract at most one server 
in each time slot (the multi-server queueing model we elaborate on throughout this paper), 
we prove that for a symmetric case with i.i.d. Bernoulli arrivals and connectivities (i.e., 
with the same arrival and connectivity parameters for all the queues), MWM not only is 
throughput optimal, but also it is optimal in minimizing, in stochastic ordering sense, a 
broad range of cost functions of queue lengths including total queue occupancy. We then 
extend the queueing model by considering imperfect services where it is assumed that the 
service of a scheduled packet fails randomly with a certain probability. We prove that MWM 
is still optimal for the extended model. We also show that the results are still valid for some 
extended connectivity and arrival processes (e.g., Poisson arrival process). 

• We discuss about the stability region of a general multi-server queueing system with general 
arrival processes with finite first and second moments. We first argue that the stability 
region form is a polytope. Then, we will introduce a linear algebraic representation of the 
stability region polytope (a linear algebraic representation equivalent to the convex hull 
representation of the stability region) which can be applied to solve utility optimization 
problems in stochastic control of such queueing systems. 
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III. Model Description 

Before we proceed to introduce the queueing model, we present basic notation often used 
throughout the paper. Other notations are introduced whenever it is necessary. All the vectors 
in this paper are row vectors. The operator " ® " is used for entry- wise multiplication of two 
matrices. By 1^ (Q/c), we denote a row vector of size K whose elements are all identically equal 
to "1" ("0"). Zero matrix of size x is denoted by {0)nxk- The expectation of random 
processes (or random variables) is denoted by E[-]. The cardinality of a set is shown by | • |. The 
closure of a set is denote by Cl{-). Finally, the boundary of a set is represented by bound(-). 

We consider a time slotted parallel queueing system with a set of parallel symmetrical queues 
J\f = {1,2,..., A^} with infinite buffer space for each queue (See Figure [T]). Packets in this system 
are assumed to have constant length and require one time slot to complete service. The service 
to this set of queues is provided through a set of identical servers namely JC = {1,2, K}. 
The connectivity of each queue n E M to each server k E )C at each time slot t is random and 
varying across time slots. We denote the connectivity of queue n to server k at time slot t by 
Cn,k{t) £ {0, 1}. At each time slot t, the connectivity state may be expressed by an A^ x if matrix 
C{t) = {Cn,k{t)) ,n E M ,k E IC, Cn,k{t) £ {0, 1}. Therefore, the connectivity process is defined 
as {C{t)}^^ with the state space C (i.e., C is the set of all A^ x A^ binary matrices). Note that C 
is a finite set with \C\ = 2^^ . The connectivity process is assumed to have stationary distribution 
with stationary probabilities P(c) = PT{C{t) = c). In fact, P(c) represents the fraction of time 
slots where the connectivity of the system is in state c. 

At any time slot, each server can serve at most one packet from a connected non-empty queue. 
Note that in the system we do not allow server sharing, i.e., a server can serve at most one queue 
per time slot. We also assume that a queue which is being serviced by a server at a given time 
slot, cannot get service from other servers during the same time slot. 

Let An{t) be the packet arrival process (number of packet arrivals) to queue n at time slot 
t. We assume that new arrivals at each time slot are added to the queues at the end of time 
slot. For these processes, assume that ElA^^t)] < Af^^^ < oo for all t. We denote the length of 
queue n at the end of time slot t (i.e., after adding the new arrivals) by Xn{t). In other words, 
Xn{t) represents the number of packets in the nth queue at the end of time slot t (or beginning 
of time slot t + 1). 
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Fig. 1 : Discrete time queueing system with parallel queues and K servers 
Definition 1: We define the multi-server queueing system of Figure [T] to be a symmetric system 

if 

• C'n,fc(i)'s are independent and E\Cn,k{t)] = P for all n E J\f and k E )C and t = 1, 2, .... 

• The arrival variables An{t) at each time slot t are i.i.d. Bernoulli random variables with the 
same parameter A for all n and t. 

A server assignment policy at each time slot determines an assignment of servers of set K, 
to the queues of set J\f. In other words, at each time slot the scheduler has to decide about 
a bipartite matching (matching in bipartite graphs) between sets M and /C. This should be 
accomplished based on the available information about the connectivities Cn,k{t) and also the 
queue length vector process at the beginning of time slot t (which is X{t — 1) = {Xi{t — 
l),X2{t — 1), ...,X7v(t — 1))). For a given policy vr, suppose that indicator variable M^l{t) is 
defined to be "1" if server k is assigned to queue n at time slot t and "0" otherwise. We define 
the N X K matrix M^''\t) = {M^^lit)),Wn E Af,k E JC as the employed matching by policy 
TT at time slot t. Therefore, a server scheduling policy tt is defined as vr = {M^'^^t)}'^^. We 
denote the matching space, i.e., the set of all the possible matchings in an x i^T bipartite graph 
by A^. In other words, 

M = Imnxk = {Mn,k) ynEAr,kElC\J2Mn,k<l G A^, J]M„,, < 1 VA; G /C I . (1) 

t k=l n=l J 

According to the above discussion, we can see that the queue length random process 
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evolves with time according to the following rule. 

X„(t)= ^X„(t-l)-^C„,fc(t)M^y2(t)j +An{t) WneJ\f (2) 

In which, (■)"'" returns the term inside the brackets if it is non-negative and zero otherwise. Note 
that a server can be assigned to an empty queue however it cannot serve it since there is no 
packet to be served. That is why we have used operator (■)+ in Q. 

As we discussed earlier, the queueing model introduced in this section is useful in modeling 
the resource assignment problem in various systems with shared resources. In wireless commu- 
nication systems, communication resources such as communication sub-channels, relay stations, 
etc. are shared among users and therefore resource assignment problem in these networks can 
be studied using our model (e.g., [[T]|, [[TtII '). For example, consider a relying access network with 

users and K relays. By modeling the cooperative wireless channel between each user, each 
relay and the base station as a binary erasure channel, we can apply the introduced queueing 
model for performance evaluation. 

In this paper, random variables are represented by CAPITAL letters and lower case letters are 
used to represent sample values of the random variables. 

In the first part of our analysis, we will consider a symmetric multi-server queueing system 
(refer to Definition [U and we prove delay optimality of Maximum Weighted Matching (MWM) 
policy for this system. In the second part of the paper, we will consider a general system 
with stationary connectivity and packet arrival processes with finite first and second moments 
for which we will characterize the network stability region polytope by a finite set of linear 
inequalities. 

IV. Maximum Weighted Matching 

In lEl, iSl, HH, [fT6l. it was shown that Back-pressure algorithm maximizes the stable through- 
put region of a general data network (i.e.. Back-pressure algorithm is throughput optimal]^. For 
the model introduced in sectionHlIl Back-pressure algorithm is equivalent to solving the following 
optimization problem at each time slot t [JJ. 

The reader is encourag ed to refer to (2), H, GH for more information regarding Back-pressure algoritiim. 
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N K 



Maximize : 



^x„(t-l)^M„,fc(t)c„,fc(t) 



n=l fc=l 



K 



Subject to: 



J]M„,,(t)<l (n = l,2..,iV) 



k=l 



N 



$^M„,fc(t)<l (A; = l,2..,i^), 



(3) 



n=l 



where x{t — 1) and c(t) are the values of the random vector X{t — 1) and random matrix C{t), 
respectively. Note that finding the solution of problem ([3]) is equivalent to finding a maximum 
weighted matching in the N x K bipartite graph Gt = (A/", IC,£) shown in Figure ^ In Gt, Af 
and JC are the two sets of vertices in each part of the graph and S = {e„,/c,Vn G J\f,yk E /C} 
is the set of edges between these two parts. Note that in Gt, the associated weight to each edge 
en,fc is Xnit — l)c„fc(t). A matching in graph Gt is a sub-graph of Gt in which no two edges 
share a common vertex. Any matching M^'^^ (t) at any time slot t is corresponding to a sub-graph 



of Gt namely Gr' = {Af,IC,£^^^) in which e„,fc G ^^'^ if and only if M2(t) = 1. Suppose 
that M(^*^)(t) = (M^™^(t)) Vn G AT, A; G /C be the matching whose indicator variables are 
the solution of the optimization problem ([3]). M^^'^^^t) can also be presented by the following 
equation. 



Consequently, we define Maximum Weighted Matching (MWM) server assignment policy as 
MWM = {M^^'^^^ (t)}'^i. There are several algorithms to find the maximum weighted matching 
in bipartite graphs. The most well known algorithm is Hungarian algorithm whose complexity 
is of 0((min{A^,A:})(max{A^,A:})2) [[2T1l. 



We first review the concepts of stochastic ordering and dynamic coupling arguments in the 
following subsection. Then, we will prove that optimality of MWM policy is beyond just 
maximizing the stable throughput region of the system. Indeed, we will prove that in a symmetric 
multi-server queueing system, MWM policy is optimal in minimizing, in stochastic ordering 
sense, a class of objective functions of queue length process. Such a class of objective functions 




(4) 



V. Optimality of MWM Policy 
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Fig. 2: Bipartite graph corresponding to problem ([3]) 

includes the total queue occupancy and therefore queueing delay. For brevity we will use the 
term "delay optimality" to refer to the optimality of MWM in this sense. 

A. Stochastic Ordering and Dynamic Coupling 

In this section, we briefly review the concepts of stochastic ordering (stochastic dominance) 
and dynamic coupling techniques. Consider two discrete time stochastic processes A = {A{t)}'^^ 
and B = {B(t)}^i in M. We say A is stochastically smaller than B and we write A <st B if 
Pr(A(t) > r) < Pr(5(t) > r) for all t = 1,2,... and all r e R (H, JH. Some properties 
of stochastic ordering are the following. If A <st B then E[A{t)] < E[B(t)]. If A <st B 
then f{A) <st f{B) for all non-decreasing functions /. A is stochastically smaller than B, 
if there exists process A = defined on the same probability space as B with the 

same probability distribution as A and satisfy A{t) < B(t) almost surely for every t = 1,2, ... 
[[TSl. The last statement is known as coupling of A and A. In fact, when applying coupling 
technique, given process A we try to construct a coupled process A with the same distribution 
as A and A{t) < B(t) as. for all t. This gives us a tool for comparing the processes A and B 
stochastically when it is infeasible to derive the distributions of A and B (e.g., in our queueing 
model when comparing the total occupancy process for different server assignment policies). 
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B. Delay Optimality of MWM 

In this section, we present one of the main results of this paper, that is proving the optimality 
of MWM with respect to minimization of a class of cost functions of queue lengths including the 
average queueing delay for the symmetric multi-server queueing system described in Definition 
[B Suppose that Z+ be the set of non-negative integers and be the dimensional Cartesian 
space of non-negative integers. We define relation " ■< " on as follows. 

Definition 2: For two vectors a; , 5; G iJ^, we write x ^ x if one of the following relations 
holds: 

Dl: Xn < Xn for all n = 1, 2, A^. 

D2: X is obtained by permutation of two distinct elements of x, i.e., x and x are different in 
only two elements n and m such that Xn = Xm and x^ = Xn- 

D3: X and x are different in only two elements n and m such that x„ < x„ < Xm < Xm and 

the following constraints are satisfied: Xn = Xn + 1 and Xm = Xm — 1- 
In D3, we say that x is more balanced than x and can be obtained by decreasing a larger 
element of x (i.e., m) by "1" and increasing a smaller element (i.e., n) by "1". We call such an 
interchange as a balancing interchange on vector x. Thus, the result of a balancing interchange 
on a vector x would be a vector x such that x < x. Suppose that vector x E Z^ represents the 
queue length vector at a given time slot. Then a balancing interchange is equivalent to taking a 
packet from a larger queue and adding it to a smaller queue. 

We define the partial order " " on Z^ as the transitive closure of relation " ^ " [[24|. In 
other words, x x if and only if x is obtained from x by performing a sequence of reductions, 
permutations of two elements and/or balancing interchanges. When x and x are two queue length 
vectors, we write x ^p x if and only if queue length vector x is obtained from x by applying a 
series of packet removals, two queues permutations and balancing interchanges. 

We define as the class of real-valued functions on that are monotone and non-decreasing 
with respect to the partial order ^p, i.e., 

/G-F ^ X ^pX ^ fix) < fix). (5) 

We can easily check that function fix) = belongs to J^. This function captures the 

total queue occupancy of the system. 
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Let X'(t) = {X[{t), X2{t), X'j^{t)) denote the queue length vector at time slot t exactly 
after serving the queues according to a server assignment policy tt and before adding the new 
arrivals of time slot t, i.e., 

X^t) = (^X^it - 1) -Y.CnAt)Mi:l{t)^ . (6) 

Given x'{t) as a sample value of random vector X'(t), we define a balancing server reallocation 
at time slot t as follows. 

Definition 3: A balancing server reallocatiorO on vector x'{t) is a matching that results in 
vector x'{t) such that one of the following conditions is satisfied. 
(CI) x'^{t) < x'^{t) for all n = 1, 2, iV and there exists m G {1, 2, A^} such that x'^{t) < 

(C2) x'{t) and x'{t) are different in only two elements n and m such that x'^{t) < x'^{t) < 
x'^{t) < x'^{t) and the following constraints are satisfied: x'^{t) = x'^{t) + 1 and x'^{t) = 

Figures [3al and [3bl show two examples of balancing server reallocations in two sample graphs. In 
these figures, the original allocations are specified by solid lines while the balancing reallocations 
are specified by dashed lines. 

Consider an arbitrary server assignment policy vr with the allocation variables {M^l{t)}'^i 
for all A; G /C and n G M. We introduce Matching Weight (MW) index associated to the server 
allocation policy tt at time slot t by 

N K 

MW.(t) = - 1) 5^c„,,(t)Ml;^(t) (7) 

n=l k=l 

Note that MW index is exactly the objective of the optimization problem ([3]). According to 
Definition [3] and definition of MW index, we can prove the following Lemma. 

Lemma 1: For a given policy tt employing matching M'^'^^(t) at time slot t, by applying a 
balancing server reallocation at time slot t (if there exists any), we will have a new policy vf 
differing from tt only at time slot t such that MW^(t) < M\N^{t). 

The detailed proof of the lemma is given in Appendix ILAi Based on Lemma [H we can state 
the following corollary. 

"'a balancing server reallocation may consist of one or more server reassignment. 
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xi{t - 1) = 3 



3\ y 



7>0 



X2(t-1) = 2 X 



X3{t - 1) = 5 ^X*^ \ 



(a) Satisfying condition CI 



Xlft - 1) = 3 



X2{t-1) = 2 



0\ Y 





























^Ti ' 



(b) Satisfying condition C2 



Fig. 3: Examples of balancing server reallocations (the weight c„,fc(t)x„(t — 1) of each edge 
(n, k) is also shown) 



Corollary 1: For a given policy tt at time slot t, if MW7r(t) is maximized, i.e., policy tt 
employs a maximum weighted matching at time slot t, then there exists no balancing server 
reallocation at that time slot. 

Note that Lemma [T] states that any balancing reallocation increases the matching weight index. 
However, it does not imply the existence of a balancing server reallocation when MW^(i(:) is not 
maximized. In the following we will prove the existence result i.e., the reverse of Lemma [U 

Lemma 2: For a given policy tt at time slot t, if MW^(t) is not maximized, i.e., MW^(t) < 
MWMWM(i), then there exists a balancing server reallocation at that time slot. 

For the detailed proof, please refer to Appendix ILB[ 

By n'^*'^, we denote the set of all policies who employ maximum weighted matching at 
all time slots. We also define as the set of all policies that employ maximum weighted 
matching exactly until time slot t (including t). We can easily observe that Tit-i ^ and 
-qmwm _ [^^^Hf. From Lemmas [H and |2l we conclude that given a policy tt G n^-i which 
is using an arbitrary matching at time slot t, we can reach to a policy tt* G by applying a 
sequence of balancing server reallocations. Suppose that represents the number of balancing 
server reallocations required to convert the employed matching in policy tt at time slot t to 
a maximum weighted matching. In this case, we say that the distance of tt from 11^ is hi 
balancing server reallocations. Note that if the distance of tt from is h^, after applying the 
first balancing server reallocation we get to a policy tt whose distance from is h1 — l balancing 



December 7, 2011 



DRAFT 



13 



server reallocations. By repeating this procedure we finally get to a policy whose distance to IIj 
is zero, i.e., it belongs to Uf By (0 < /i < h^) we denote the set of all server assignment 
policies in n(„i whose distance from is at most h balancing sever reallocations. Note that 

n? = n,. 

Consider any two policies vr and vf such that f{X) <st f{X) where X = {X(t)}'^i and 
X = {X(t)}^]^ are the queue length processes when policies tt and n are applied, respectively. 
For such a system, we say policy tt dominates tt. If vf dominates tt we have E[f{X)] < E[f{X)^. 
Based on the above discussion about stochastic dominance, we can prove the following Lemma. 

Lemma 3: For any policy tt G nj* and < h < we can construct a policy tt G H^^^ such 
that TT dominates tt. 

For the detailed proof, please refer to Appendix II-CI Using Lemma [3l we can prove the main 
result of this section in the following Theorem. 

Theorem 1: The Maximum Weighted Matching policy dominates any server assignment pol- 
icy. 

Proof: Let tcq be any arbitrary policy. Then ttq G Ho = Ilf^^ where Hi = h\° . By applying 
Lemma [3] repeatedly, we can construct a sequence of policies such that each policy dominates 
the previous one. Thus, we obtain policies that belong to Ho = nf^^, Ilf^^"^, Ilf^^"^, = Hi. 
The last policy is called tti. Note that tti G 11^^ where H2 = h^^. By recursively continuing 
such an argument we obtain a sequence of policies ixt G lit, t = 1, 2, ... such that tTj dominates 
TTj for j > i. Note that this sequence of policies defines a limiting policy tt* that agrees with 
MWM at all time slots. Thus, tt* is an MWM policy who dominates all the previous policies, 
including the starting policy ttq. ■ 

C. Extensions 

1 ) Imperfect Services: We can extend Theorem [T] for the case where the service of a scheduled 
packet by a connected server fails randomly with a certain probability. This is in analogous to 
realistic wireless networks operation where service failures usually occur due to unexpected and 
unpredictable effects of noise, interference, etc. In case of a packet service failure, the packet 
will be kept in the queue and will be rescheduled and retransmitted in future time slots. 

^Choosing f{x) — X^^ILi conclude that the average queue occupancy (or equivalently average queueing delay) of 

policy TT is smaller than that of policy tt. 



December 7, 201 1 



DRAFT 



14 

We denote the binary random variable associated to successful/unsuccessful service of queue 
n provided by server k at time slot t by Qn,k{t)- Note that Qn,k{t) G {0, 1} and is "1" ("0") when 
the service is successful (unsuccessful). We assume that Qn,k{ty^ are i.i.d. Bernoulli random 
variables with the same success probability q. Note that parameter q (the same as parameters 
A and p) does not involve in our analysis and in fact what is necessary in our analysis is the 
symmetry of the system (i.e., E[Qn,k{t)] = q,yn, k). The queue lengths are then updated at the 
end of each time slot by the following rule. 

Xn{t) = ||x„(t - 1) - I] C'„,,(t)M(5(t)Q„,fc(t) j + A^{t) \fneAf (8) 

Note that the network controller cannot observe the variables Qn,k{t) and from his perspective 
they are assumed to be random. The random vector X'{t) is defined similar to equation 
Hence, X'(t) represents the queue lengths before adding the new arrivals of time slot t as if all 
the services at that time slot are successful. 

For such a system we can easily see that Lemmas [T] and [2] are still valid. We can also extend 
Lemma [3] for the system with service failures by considering random variables Qn,k{t) in our 
dynamic coupling argument. The detailed analysis is brought in Appendix ILDI By applying the 
same approach as what we did for Theorem [H we can similarly prove the delay optimality of 
MWM policy for the system with imperfect services. 

2) Extensions for Connectivity and Arrival Processes: The arguments in Lemma [3] and The- 
orem [T] remain valid if the i.i.d. assumption for connectivity and arrival processes is relaxed as 
follows; we will consider connectivity and arrival processes which follow conditional permutation 
invariant distributions. Given even 1-L (which is usually used to denote the history of the system), 
we define a conditional multivariate probability distribution f{yi,y2,---,yn I to be permutation 
invariant if for any permutation of the variables ?/25 1/n namely y[,y2, .■■,yn we have 
f{yi,y2,...,yn \ 'H) = f{y[,y'2,...,y'n \ 'H). We can readily see that for all the connectivity 
and arrival processes whose joint distributions at each time slot given the history of the systerr^ 
(i.e., fA{t)iai,a2,-,aN I "H) and fc(t)ici^i, Ci^2, cn,k-i, cn,k I "H)) are permutation invariant. 
Lemma [3] and Theorem \T\ are still valid and therefore MWM is delay optimal. 

*By history of the system we mean all the channel states, arrivals and matchings of the previous time slots. 
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We also consider the generalization of Theorem [T] for non-Bernoulli arrival processes. Suppose 
that the number of arrivals to each queue can be represented by the summation of some i.i.d. 
Bernoulli random variables, i.e., has Binomial distribution. Also suppose that v4„(t) < A„iax far 
all n E M and all t. In this case, we can create a new (virtual) system in which after each time 
slot we append A^ax — 1 virtual time slots and put the connectivities all equal to zero, i.e., for 
each virtual time slot t, Cn,k{t) = 0,Vn G J\f,yk E JC. We then distribute the arrivals of the 
actual time slot among these Amax time slots (one actual time slot and Amax — 1 virtual time slots) 
randomly such that at each time slot at most one packet arrival occurs. Since the connectivities 
and the arrivals in both systems are permutation invariant, we can still prove Theorem [U for the 
virtual system. Note that the operation of the two systems (the original system and the virtual 
system) are the same. Therefore, we can conclude that Theorem [T] is also valid for a multi-server 
system with Binomial arrival processes. As Poisson distribution asymptomatically approximates 
the Binomial distribution when Amax goes to infinity. Theorem \T\ will be still valid for i.i.d. 
Poisson arrival processes. 



In this section of the paper, we derive a linear algebraic characterization of the stability region 
for the queueing system of Figure \T\ Linear algebraic representation of the stability region is 
specially useful in solving stochastic optimization problems and finding optimal flow control 
strategies in queueing networks. 

A. Motivation 

Stability region or network stability region of a communication network is the closure of the 
set of all arrival rate vectors for which there exists a resource allocation policy that can stabilize 
the system [|2l, flU, [fTSl . Having the stability region A of a network characterized by a set of 
convex inequalities, we can obtain a precise solution for network utility optimization problems 
with the general form of (0). 



VI. Linear Algebraic Characterization of Stability Region 



TV 



Maximi ze : 




(9) 



n=l 



Subject to: 



r = (ri,r2, ...,rAr) E A 



0<r„<A„ Vn = l,2,..., 



N 
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In^, N is the number of traffic sources, r = (ri, r2, r^r) denotes the admitted rate vector, A„ 
is the input rate of each source n and A denotes the network stability region. Functions /„(r„) 
are non-decreasing and concave utility functions. The choice of /n(r„) determines the desired 
fairness behavior of the network ||251 , ||26l . 

To solve the optimization problem we need to have a precise description of stability 
region A. Although stability region is already defined and characterized in [l2l-[|4l|, [[T2l|. 
ifTSl . [fT6l as the convex hull of a set of fixed points, such a description cannot be applied to 
solve problems similar to ^ as it does not provide explicit mathematical relations in the form of 
convex equalities or inequalities to put as the constraints of problem In this section, we will 
derive a linear algebraic representation of the stability region of multi-server queueing system 
of Figure [T] 

B. Preliminaries 

1) Stability Region and Throughput Optimal Server Assignment: The concepts of stability 
region and throughput optimal resource allocation in queueing networks were first introduced in 
the seminal work O, |l3l by Tassiulas and Ephremides. The idea was further developed by Neely 
et al. for more general queueing networks in [fT5l . [fT6ll . We first review the definition of strong 
stability in queueing networks H. Stability region is then described based on this definition. 
Consider a discrete time single queue with an arrival process A(t) and service process Let 
X{t) denote the queue length process at the end of time slot t. Hence, X{t) evolves with time 
according to the following equation. 



Definition 4: A single queue evolving according to (flOl) is called strongly stable if it has 
bounded time averaged expected backlog flU, i.e.. 



Definition |4] is extended to define stability in a queueing network as follows H. 

Definition 5: A queueing system is called to be strongly stable if all the queues in the system 
are strongly stable. Specifically, for a network consisting of N queues, the system is strongly 



X{t) = (X(t-l)-^(t))+ + A(t) 



(10) 




(11) 
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Stable if the time averaged expected aggregated backlog in the network is bounded, i.e., 

^ t-l N 

limsup-^^E[X„(r)] < oo. (12) 

^ r=0 n=l 

Consider the queueing system introduced in section UlI] (Figure [U). Assume that the arrival 
rate to each queue n is denoted by A„ and therefore the arrival rate vector is denoted by A = 
(Ai, A2, Aat). A server assignment policy vr is said to stabilize the system if by applying policy 
TT the system is strongly stable. Let represent the set of arrival rate vectors for which tt can 
stabilize the system. A^^ is called the stability region of policy tt. Now, consider the closure 
of the union of the stability regions of all the possible server assignment policies and let us 
denote it by A, i.e., A = C/(|J^A^). A is called network stability region (also called network 
layer capacity region in the literature JH, lfT3l - [fT6ll ). Therefore, network stability region is the 
closure of the set of all arrival rate vectors for which there exists a policy that can stabilize 
the system. We observe that the network stability region is independent of resource allocation 
policy and is in fact a specific characteristic of any network. In section IVI-C[ we will describe 
the characterization of stability region as the convex hull of a set of fixed points in (i.e., a 
convex poly tope) introduced in ||4l, |I71, ifTSl . 

2) Fundamental Concepts of Polytopes: In this part, we briefly review the notion of convex 
poly tope and some of its fundamental properties. These concepts are needed when describing 
the stability region of the system in sections IVI-CI and IVI-DI 

Definition 6: A convex poly tope is defined as the convex hull of a finite set of points [271 . 

m. 

According to the Weyl's Theorem [|27l . a polytope in always can be expressed by a set 
V = {x E I agx^ < , ^ = 1,2, for some positive integer L and ag E and 

A E R. 

Definition 7: An equality ax^ < 6 is called valid for a polytope V if for every point xq E V, 
axQ < b. 

Definition 8: A face of polytope V is defined as J-' = {x E V \ ax'^ = b} where inequality 
ax'^ < 6 is a valid inequality for V. 

Note that if the valid inequality ax^ < b does not touch polytopre V, the associated face to 
this inequality is empty set 0. We call the hyperplane axg = b associated to the valid inequality 
axQ < b a face defining hyperplane for V if its associated face is not empty. In other words. 
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axQ = 6 is a face defining hyperplane for V if it touches V at least at one point. Note that V 
has finitely many faces. However, the face defining hyperplanes of a polytope can be infinite. 

Definition 9: A facet of polytope V is a maximal face distinct from V [|28l . In other words, 
all the faces of V with dimension dim(7') — 1 are called facets of T\!_. 

For polytope V = {x E | a^x^ i = 1,2, an inequality is redundant if poly- 

tope V remains unchanged by removing the inequality. Redundancy theorem in polytopes states 
that to describe a polytope, only facet defining hyperplanes are sufficient [|27l . 



C. Convex Polytope Representation of Stability Region 

Consider the class of deterministic server assignment policies Q in which at each time slot 
t, a policy g E Q assigns the servers to the queues according to a predetermined matching 
depending on the connectivity state of the system (i.e., matrix C{t)). More specifically, for 
each policy g E G, there exists a one to one mapping from the connectivity matrix space C 
to the matching space ^A, namely M^^^ : C i — > Ai. At each time slot t, the scheduler who 
is employing server assignment policy g observes the connectivity matrix of the system, i.e., 
c(t) E C and assigns the servers to the queues according to matching M^^\c{t)). Note that since 
< oo and \C\ < oo, the set Q of all the deterministic server assignment policies is finite. 

Each deterministic policy g provides an average service rate for each queue n. Let Rn^ denote 
the averaged service rate provided to queue n and i?^^) = {Ri\R2 \ ...,R^^) is the vector of 
average service rates provided by policy g. By conditioning on C{t), we can readily show that 
for each deterministic server assignment policy g, R^^^ is obtained by the following equation. 

^^'^ = {^^^""^ (c®M(^)(c))ljj (13) 

Note that each rate vector R^^^ determines a single point in M^. Now, consider the convex 
hull of all the points R^^)^ g eQ in M^, i.e., 

TZ = conv.hull{R^^^). (14) 

'For a polytope V C R^, dimension of P is equal to A'' minus the maximum number of linearly independent equations 
satisfied by all the points in V. 
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Set TZ contains all the achievable service rate vectors for the queueing system of Figure \T\ Each 
point R* in TZ can be represented by a convex combination of a finite set of points, {R^^\g E Q}, 
i.e., 

= Y,P^R'' ,9^^Q , J2p' = 1> P« > 0- (15) 

i=l i=l 

Hence, in order to achieve an arbitrary service rate vector R*, it is enough to select policy gi 
in Pi fraction of time slots (a randomized policy). According to the definition of convex poly tope 
ETl . [|28l and the fact that set Q is finite, set TZ represents a convex polytope in R^. Based 
on the definition of stability region and the analysis in BU, [fTSl . [fT6l . we can conclude that 
polytope TZ specifies the stability region of the queueing system of Figure [Tl Specifically, we 
can show that if the system is stable, we should have \ E TZ and also if A G 7^ — bound(JZ), 
then there exists a server assignment policy that can stabilize the system. 

A server assignment policy n* is called throughput optimal if it can stabilize the system for 
all the arrival rate vectors strictly inside the network stability region. Therefore, tt* is throughput 
optimal if its stability region coincides with the network stability region. The MWM policy is 
proven to be a throughput optimal server assignment policy, i.e., it will stabilize the system as 
long as the input rates are inside the stability region. Note that MWM does not need to know the 
stability region to act. However, for the cases where the input rates are outside the stability region, 
in order to guarantee a measure of fairness among the admitted rates of different users, we need 
to solve network utility optimization problems similar to Q. To solve such network optimization 
problems precisely, we need to have an explicit characterization of network stability region in 
terms of linear or non-linear convex equalities and inequalities to be used as the constraints of 
problem 

D. Linear Algebraic Representation of stability region 

In the following, we will characterize the stability region of the queueing system of Figure [T] 
by a finite set of linear inequalities. We introduce the departure matrix HN^Kit) = (if„ jt(t)), n E 
J\f,k G /C in which Hn,k{t) represents the number of packets served by server k from queue 
n at time slot t. Obviously Hn^k{t) & {0, 1} and i7„ ^(t) < Cn,k{t) < 1- Hence, the departure 
process from queue n at time slot t can be written as J2k=i ^n,k(t) ■ The following equation 
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describes the evolution of the queue length process in time. 



K 



X„(t) = + Mt) ^neM (16) 

fc=i 

For a strongly stable queueing system the following is true. 

Lemma 4: If the queueing system of Figure [U is strongly stable under a server assignment 
policy TT, then for any vector a = (ai, 02, ^Af) G we have 



t 



aX' = lim - V aE[H{T)]ll. (17) 

T = l 



The proof is given in Appendix III-A[ 

In other words, for a strongly stable system, the weighted arrival rates to the queues is equal 
to the time averaged weighted expected departure from the queues. We further can prove the 
following Lemma. 

Lemma 5: If the queueing system of Figure [U is strongly stable under a server assignment 
policy TT, then for all a G 

aX^ < V P(c) max (a(c ® M)ll) . (18) 

c&C 

The proof is given in Appendix III-B[ 

As we can see, the set of inequalities in (fTSi) is an infinite set. Note that each inequality in (flSl) 
determines a valid inequality for polytope IZ. It can be shown that the hyperplanes associated 
to the valid inequalities of (fTSi) are all face defining hyperplanes of polytope IZ. Therefore, the 
hyperplane associated to each of the valid inequalities of (fTSi) touches polytope IZ at least at 
one point. Towards this, let A^" denote the set of matchings {M"(c),c G C} that maximizes 
the right hand side of (fTSi) . i.e., 

M"(c) = arg max a{c ® M)l^ (19) 

Note that A^" is not unique and there may be more than one set of matchings whose 
elements maximize a(c ® M)lJ. According to ^ and d), [Y.cec ^(c) (c ® M°(c)) 1 J)^ G 
7^. Furthermore, since Ecec ® ^"(c))!^ = " Ecec ^(c)(c ® M"(c))l^, according to 

(fTSl ) this point (i.e., (XIcgc -^('^) ('^ ® ^"(c))ii-)^) is located on the hyperplane associated to 
(fTSi) . Hence, the set of inequalities of (fTSi) determines all the non-empty faces of polytope IZ. 
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According to redundancy theorem in poly topes (refer to Section IVI-B2I and also [|27l . [|28l ). 
not all the face defining hyperplanes of a polytope are necessary to describe a polytope; i.e., the 
inequalities corresponding to the facets of a polytope are sufficient to characterize a polytope. In 
the following theorem, we will show that not all a E are required to describe polytope TZ. 
In fact. Theorem [2] states that just a E {0, 1}^ — {Ojy} are sufficient to describe facet defining 
hyperplanes of polytope TZ. 

Theorem 2: If there exists a server assignment policy tt under which the system is stable, 
then 

< P{c) max (a(c ® M)l^) , a e {0, 1}^ - {O^v}- (20) 

cec 

The proof is given in Appendix |II-C[ 

Note that Theorem |2] describes the necessary conditions for the stability of the system. For 
completeness of our study we also have to show that all the rate vectors inside polytope TZ are 
rate stable. In other word, there exists a server assignment policy who can stabilize the system 
for all the rate vectors inside polytope TZ. In the following theorem, we will prove that for a 
system satisfying conditions (|2TI) below (which is describing TZ — hound(TZ)), the MWM policy 
stabilizes the system. We further obtain an upper bound for average expected aggregate backlog 
in the system. 

Theorem 3: The multi-server queueing system of Figure [T] is stable under MWM if 

aX^ < P{c) max (a(c ® M)l^) , a G {0, 1}^ - {0^}. (21) 

Furthermore, the following bound for the average expected "aggregate" occupancy holds. 

lim.up i £ f E[XAr)] < ^■^^ + <T'^' ^'"^ (22) 

i^oo t ^ ^ 26 

T=0 71=1 

where 5 is a positive real number such that A + 51^ E hound{TZ). 

The proof of Theorem |3] follows by applying Lyapunov stability analysis introduced in [^, 
[fTSl . [[T6l . For completeness of our analysis the proof is provided in Appendix III-DI 

According to Theorem [2] and |3] and the definition of network stability region, we can conclude 
that for the multi-server queueing system of Figure [U with stationary connectivity distribution 
and stationary arrival processes, the stability region is characterized by (|20|) . 
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VII. Simulation Results 

We have evaluated the performance of MWM policy with two other server assignment policies 
in terms of average queueing delay. The other two policies are described in the following. 

• Maximum Matching (MM) policy that applies the maximum matching on matrix C{t). In 
other words, the maximum matching policy at each time slot t employs a server assignment 
(or matching) M^^^\t) which is obtained by M^^^\t) = argmaxA/g^ (c(t) ® M)l^. 

• A heuristic policy that assigns the servers to the queues at each time slot according to the 
following rule: It selects a server randomly and assigns it to its longest connected queue. 
Then, updates the set of servers by removing the selected server from JC and the set of 
queues (i.e., J\f) by removing the queue to which the selected server was assigned. This 
procedure is repeated K times. Note that, for some servers the updated set J\f may be empty 
(e.g., when K > N) and therefore those servers are not assigned to any queue. 

Algorithm 1 Heuristic Policy Pseudocode 
input: M, /C, c(t) and x{t — 1) 

initialize: M^^\t) = {0)nxk 

for i = 1 to do 

Choose a server k* E IC randomly 
if Ar^0 

n* < — argmax„g^c„,fe*(t)a;„(t - 1) 

AT ^ A/" - {n*} 
endif 

JC^IC- {k"} 

end 

Return M^^\t) 



The main idea of the heuristic policy is coming from Longest Connected Queue (LCQ) policy 
which was proven to be optimal for a single server system Q. For multi-server system, we 
will use the same principle for each server. However, the order in which servers are selected 
for assignment is random. We have arranged a comprehensive set of simulations in which we 



December 7, 2011 



DRAFT 



23 



investigate the effects of number of servers K, probability of connectivity p and probability of 
service success q on the performance of the aforementioned policies. In all the simulations, we 
set = 8 and the arrivals are i.i.d. Poisson distributed. 

Figures 11116] illustrate the simulation results. As we can see in all the cases, MWM has better 
performance with respect to other policies in terms of average queue occupancy or average 
queueing delay. Figured shows the simulation results for K = A, q = 0.8 and p = 0.2, 0.5, 
0.8. In these cases, since the number of servers is relatively low, server assignment will be more 
competitive. MWM which was proven to be the optimal policy performs such that it tries to 
balance the queues. The heuristic policy follows the same principle. However, since the selection 
of servers for assignment is random there are some cases where two or more servers have the 
same longest connected queue. In such cases, the order of selecting the servers for assignment 
is effective in the performance of the system. Maximum Matching policy however, does not try 
to balance the queues. Indeed, it does not consider the queue lengths in its assignments and 
that is why it performs worse than the other two policies. Note that as the connectivity of the 
system gets larger, the performances of MWM and the heuristic policy get closer. It is worth 
mentioning that the heuristic policy introduced here performs the same as MWM for K = 1 
(which is equivalent to LCQ whose optimality was previously shown in [3J). 

Figure 13 shows the results for 8 servers. In this case, since the number of servers is relatively 
high and comparable to the number of queues, in MWM and MM policies each queue gets 
service with high probability when the probability of server connectivities tends to 1. As the 
connectivity probability gets smaller, the difference in performance of MWM and MM becomes 
more apparent. Note that in this case, the heuristic policy performs worse than the other two 
policies since it is more probable to lead to cases where two or more servers are connected to 
the same longest connected queue. As the number of severs becomes larger, we expect MM to 
perform the same as MWM as in this case the probability of serving all the queues increases. 
Therefore, in the limiting case where K becomes very large, MM and MWM result in similar 
performance. 

In Figure [6l we have investigated the effect of service success probability. As we can see in 
the figures, the only effect of this parameter is to change the stability point (the arrival rate at 
which queue occupancy tends to infinity). In this case, again we can see that for all q = 0.2, 
0.5, 0.8 MWM policy performs better than the other policies. 
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VIII. Conclusions 

In this paper, we considered the problem of assignment of K identical servers to a set of 
parallel queues in a time slotted multi-server queueing system with random connectivities. 
For such systems, it has been previously shown that MWM is throughput optimal, i.e., has the 
maximum stability region. In this paper, we first showed that for a symmetric system (refer 
to Definition [U MWM is also optimal in minimization of a class of cost functions of queue 
lengths including the average queueing delay. Our approach in the proof was based on stochastic 
ordering and dynamic coupling techniques. Second, we derived a linear algebraic representation 
of the stability region for the aforementioned multi-server queueing system. We showed that the 
stability region form is a convex polytope for which we explicitly determined the coefficients 
of the facet defining hyperplanes associated to the stability region polytope. 

Appendix I 
Proof of Lemma [B Lemma [2] and Lemma [3] 

A. Proof of Lemma [7] 

Proof: Let M^'^^ (t) denote the employed matching after applying the balancing reallocation. 
According to the definition of balancing server reallocation, a server reallocation at time slot 
t results in a queue length vector x'{t) that satisfies either condition CI or C2. Therefore, we 
consider the following two cases: 

Case 1: Condition CI is satisfied at time slot t. Thus, x'^{t) < x'^(t) for all = 1, 2, and 
there exists m E {1,2,...,A^} such that < x'^{t) < x'^{t). Therefore, there exists no queue 
that was served by tt but not by if. Also queue m which was not receiving service by vr at 
time slot t, is now receiving service after applying the balancing server reallocation. Therefore, 
^mit - 1) Ef=i CmAt)Mi:lit) = 0, xUt - 1) Ef=i c™,fc(t)M« (t) = x^t - 1) > and 

N K N K 

J2 Xn{t-l)Y,CnAm'alit)< E " 1) E ^-''^ ^^^^^ W " ^^^^ 

n=l,n^m k=l n=l,n^m k=l 

Therefore, MW^(t) < MW^(t). 

Case 2: Condition C2 is satisfied at time t. In this case, by using policy tt at time slot t queue 
n is receiving service but queue m is not. In contrast, by using policy tt, at time slot t queue 
m is receiving service but queue n is not. Therefore, we have 

MW^(t) - MW^(t) = Xm{t - 1) - Xn{t - 1) > (24) 
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Therefore, MW^(t) < M\Nji.{t). By considering cases 1 and 2 the lemma follows. ■ 
B. Proof of Lemma |2] 

Proof: Without loss of generality, we may convert the bipartite graph Gt to a complete 
weighted bipartite graph G[. This is done by adding some vertices and edges of zero weight as 
necessary. In particular, if > i^, we will add N — K servers on the right hand side with edges 
of weight "0" to each queue (each vertex on the left hand side). It is like the case that we add 
some servers with service rate to the system. \f N < K, we will add K — N queues on the left 
hand side with edges of weight "0" to each server (each vertex on the right hand side). It is like 
the case that we add some queues which are disconnected from all the servers. Thus, we deal 
with a complete bipartite graph with max{A^, K} vertices on each side. We denote the sets of 
vertices on each part of G'^ by J\f' and /C', respectively and the set of edges by £'. Consequently, 
a policy vr is defined as tt = {MW(t)}^i where MW(t) is a perfect matching in the complete 
bipartite graph G[. Note that a maximum weight perfect matching M^^^^\t) in the complete 
bipartite graph is the same as the maximum weighted matching in graph Gt if we remove 
the added edges of weight "0" from matching M('^*'^)(t). 

Consider now a policy vr which is employing perfect matching M^'^'> (t) at time slot t. Suppose 
that M'^'^^ (t) is not a maximum weight perfect matching on graph G^, i.e., MW7r(t) < MW]viwM(i)- 
Also, consider a maximum weight perfect matching M^'^*'^^ (t) at time slot t. Now, consider these 
two matchings on G[ = {J\f', )€',£'). Each of M('')(t) and M(^*^)(t) corresponds to a distinct 
sub-graph of namely G'S"^ = (M' , IC' , S'^^y) and G"J™^ = (A/"', /C', f '(™)), respectively. 
We now build two directed weighted sub-graphs Dt""^ and d^^"^^^ as follows: dI"^^ is the same 
as G'l'^^ with all the edges directed from A/"' to /C' with the same edge weights as G'^^^K £)(mwm) 
is the same as q'^^"^^^ ^ith all the edges directed from /C' to A/"' with edge weights equal to the 
negative of edge weights of q'^^^^\ Now, consider graph U = D^"^^ [JdI^^^\ i.e., the union 
of the sub-graphs and Graph U is also the union of some even cycles denoted by 

L. This is directly concluded from the fact that 0^"^^ and d^^^^^ are each perfect matchings 
of G't and thus each vertex is incident to an incoming edge and an outgoing edge. Furthermore, 
for the weight of U we have 

w{U) = ^w(£) = MW^(t) - MWmwmW < 0. (25) 

eeL 



December 7, 2011 



DRAFT 



29 




Therefore, there must exist a negative cycle in U namely i*. Cycle i* is an even cycle that 
contains an even number of nodes let say 2W nodes (W nodes from J\f' and W nodes from 
/C') and also 2W edges. Let us denote the nodes of set J\f' and K,' that contribute in by 
ni, 77-2, nw and ki, kw, respectively. Thus, cycle i* can be represented by the sequence 

of its edges as ^ Cni,A;i; Cfc]^^n25 ^n2:'s25 *^fc2i"35 ^fcw-ii"w ^'^'Wi^iv ^'^vk:'*! (^^^ Figure |7]). 

Note that edges Cfc^^^j) 6A;2,n3; e^^^n^ belong to jj^^"^^^ and have negative weights while 
edges eni,ki, en2,k2^ ■■■^^nw,kw belong to Dl and have positive weights. We can also represent 
the edges of i* by pairing the edges incident to each node ni,n2, ■■■,nw as follows; £* = 
{ekw,ni, (^mM)^ i(^kun2, en^M)^ ■■■A(^kw-i,nw^ (^nw,kw)- Wc label cach pair of edges as follows: 

pairs in £* have the following property: The weights associated to the edges of each pair r„^, 
l<i<Wis either (0,0), (0, x„^(t-l)),(-x„,(t-l), 0) or (-x„,(t-l), Accordingly, 
we will specify three types of edge pair as follows: 

. Type (TO): pairs with edge weights (0,0) or (-x„^(t - l),a;„,(t - 1)), l<i<W . 

. Type 1 (Tl): pairs with edge weights (-x„^(t - 1),0), 1 < i < W and Xn^t - 1) > 0. 

. Type 2 (T2): pairs with edge weights (0,x„^(t - 1)), 1 < z < and x„,(t - 1) > 0. 

Since is a negative cycle, there must exist a node nj with edge pairs {ek^_^^nj-,&nj,kj) = 
{-Xn.it - 1),0) (if i = 1, (efc^y,„,,en,,fci) = {-x^it - 1),0)) and Xn^{t - 1) > 0. Obviously, 
the edge pair associated to node nj is of type Tl. We now consider the following three cases: 
Case 1: All the edge pairs other than are of type TO. In this case, by replacing the edges of 
D^'^^ by the edges of d^^'^^^ in the negative cycle £* we obtain a server reallocation at time 
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(a) Case 2 (b) Case 3 

Fig. 8: Cases 2 and 3 in the proof of Lemma |2] 



slot t. This server reallocation is balancing as in vr queue rij was not receiving service while 
after the server reallocation it is. Therefore, if x'{t) be the queue length vector after server 
reallocation we have x'{t) < x'{t) (satisfying condition CI). 

Case 2: If we go backward on the edge pairs of cycle i* from and the first non-TO pair is 
a Tl pair. Let r„^, be such a Tl edge pair. In other words, in cycle t the pairs r„ , and r„^. 
are of type Tl while other pairs between them are of type TO (see Figure (Sa]). In this case, 
by replacing the edges of D^,^'' by the edges of d^^^^^i just for nodes rij'+i to rij of cycle 
and allocating kj to riji we obtain a server reallocation at time slot t. This server reallocation 
is balancing as in tt queue rij was not receiving service while after the server reallocation it 
is and also queue riji was not being serviced in tt but now may get service (depending on the 
weight of edge e^^ „ ,) and the service of other queues is not disturbed. Therefore, if x'{t) be 
the queue length vector after server reallocation, we have x'{t) < x'{t) and x'^.it) < x'^^{t) 
(satisfying condition CI). 

Case 3: If we go backward on the edge pairs of cycle from and the first non-TO pair is 
a T2 pair. Let r„ , be such a T2 edge pair. In other words, in cycle pair r„ , is of type T2 
and is of type Tl while other pairs between them are of type TO (see Figure [8bl). First of 
all, we claim that Xn^, — 1) < (t—l). If Xn^, — 1) > (t — 1), by replacing the edges of 
^(MWM) edges of D^'^'^ just for nodes nj' to rij^i of the cycle and not serving queue 
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rij, without disturbing the service of other queues we obtain a server reallocation at time slot 
t with larger MW index than MWmwm(^) and this contradicts with the fact that MWM has the 
maximum MW index at time slot t. Accordingly, we consider the following two sub-cases: 
Sub-case 3.1: — 1) > Xn.,{t — 1). In this case, we replace the edges of dI""^ by the 
edges of just for queues Uf+i to rij of the cycle £* and allocate server kj to queue 

rij/. Note that server kj may or may not serve queue rij'. We consider the worst case where 
it does not serve this queue (wn.,,kj{t) — 0). Thus, without disturbing the service of other 
queues we obtain a server reallocation with the following property: all the queues other than 
rij and nj' have the same service as before; queue uj which was receiving zero service in 
TT, now receives one packet service and queue n-,/ which was receiving one packet service, 
now in the worst case looses its service. Therefore, this server reallocation is balancing as the 
queue length after server reallocation (denoted by x'{t)) and the queue length after applying 
policy TT (denoted by x'{t)) satisfy condition C2. So, x'{t) ^ x'{t) and therefore the applied 
server reallocation is a balancing one. 

Sub-case 3.2: Xn^{t — 1) = (t — 1). In this case, the sequence of edge pairs r„ , , ...,r„^ 
contribute "0" in the calculation of w{U). Therefore, we may treat them as a sequence of 
edge pairs of type TO. By doing so, we have a new negative cycle £'* which is the same as £* 
but the sequence of edge pairs r„^,, are replaces by the same number of edge pairs of 
type TO. Note that cycle £'* is a negative cycle with the same weight as i*. Therefore, there 
must exist an edge pair of type Tl in £'* (In this case, it is not possible for £'* to have just 
edge pairs of type TO as it results in w{i'*) = 0) and all the cases of 1, 2 and 3 applies for 
as well. Note that as long as case 3.2 is true, we always obtain a new negative cycle and 
the whole process repeats. 
Cases 1, 2 and 3 cover all the possible cases of negative cycle £* for all of which we proved 
that there exists a balancing server reallocation. Putting everything in a nutshell, we proved that 
if the policy n is not maximum weighted matching at time slot t there exists a negative cycle 
in the graph obtained by the taking the union of the matchings of vr and MWM (i.e., U). We 
proved that by reallocation of the servers involved in the negative cycle, we always can find a 
balancing server reallocation for policy tt. ■ 
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C. Proof of Lemma \3\ 

Proof: Fix any arbitrary policy vr G IlJ' and any arbitrary sample path 

u = (x(0), c(l), a(l), x(l), c(2), a(2), x(2), ...) 

of the underlying random variables (X(0), A(l), C(2), A(2), X(2), ...)• We apply 
the coupling method to construct from u a new sample path 

u = (x(0), c(l), a(l), x(l), c(2), S(2), x(2), ...) 

resulting in a new sequence of random variables (X(0), C'(l), A(l), X(l), (5(2), A(2), X(2), ...) 
with X(0) = X(0). Note that X(0) is the queue length vector by which the system starts 
working. We denote the policy driven by the new sample path Co by tt. In fact, we construct Co and 
vr G n^~^ in such a fashion that /or all the sample paths and all time slots we have x{t) x{t). 
Therefore, for any f E T we have f{x{t)) < f{x{t)) for all t. Since processes {(C(t), A(t))}^]^ 
and {((^(if:), yl(t))}^]^ are the same in distribution (these processes are permutation invariant), 
the process {/(X(t))} obtained by applying policy ?t to the system is stochastically smaller than 
{/(X(t))}, i.e., {/(X(t))} <,t {f{X{t))} and ^ dominates vr. 

Therefore, in the following, our goal would be to construct vf and Co such that x{t) x{t) is 
satisfied for all time slots. In the proof, we always use the tilde notation for all random variables 
that belong to the new system. The construction of vf is done in two steps: 

Step 1: Construction of n for r < t: To construct the new sample path u we let the arrival, 
connectivity and the policy be the same as the first system until time slot t—1, i.e., c(r) = c(r), 
a(r) = a(r) and M*^'^^(r) = M*^'^^(r) for r < t — 1. Thus, the resulting queue lengths at the 
beginning of time slot t (or at the end of time slot t — 1) are equal, i.e., x{t — 1) = x{t — 1). We 
now consider the construction of ij and vf for time slot t. For this time slot, one of the following 
two cases may apply. 

1) The distance of policy vr to is less that h balancing server reallocations. Therefore 
71 G n^^"^ and we let c(t) = c(t) and d{t) = a{t) and M(*)(t) = M^'"\t). Therefore, x{t) 
and x{t) are equal in both sample path uj and Co. Thus, x{t) ■< x{t). 

2) The distance of policy vr to is exactly h balancing server reallocations. Since vr G nj* 
and /i > 0, according to Lemma [2] there exists a balancing server reallocation such that 
either CI or C2 is satisfied. Thus, we consider two cases: 
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Case 2.1: After applying the balancing reallocation, condition CI is satisfied. In other words, 
there exists a matching such that if applied on the queue length x{t — 1) = x{t — I) aX time 
slot t, we get x'{t) such that x'{t) < x'{t). We call such a matching by M^^'^t). In this 
case, we let c{t) = c{t) and d{t) = a{t) and we apply M^''\t) at time slot t, i.e., arrivals 
and connectivities are the same in both systems and policy tt acts at time slot t. So, we 
can easily check that x{t) < x{t) and therefore x{t) ^ x{t). 

Case 2.2: After applying the balancing reallocation, condition C2 is satisfied. In other words, 
there exists a matching such that if applied on the system at time slot t, we get x'{t) which 
is different from x'{t) in two elements m and n such that x[^(t) < x'^{t) < x'^{t) < x'j^(t) 
and the following constraints are satisfied: = x'^{t) + 1 and 5;^j(t) = x'^Jt) — 1. We 

call such a matching by M^'^\t). In this case, we let c{t) — c{t) and a{t) = a{t) and we 
apply M^^\t) at time slot t, i.e., arrivals and connectivities are the same in both systems 
and policy n acts at time slot t. We consider all the following conditions for arrivals to 
queues m and n as follows: 

• If there is no arrival or there is an arrival to both queues m and n (i.e., a„i{f ) = a,n{t) = 
or amit) = an{t) = 1), we conclude that x{t) and x{t) satisfy condition D3. Thus, 
x{t) ^ x{t). 

• If there is an arrival to queue m but not n (i.e., a^(t) = 1, a„(t) = 0), we conclude 
that x{t) and x{t) satisfy condition D3. Thus, x{t) x{t). 

• If there is an arrival to queue n but not m (i.e., am{t) — 0, an{t) — 1) and Xm{t) — Xn{t), 
we conclude that x{t) and x{t) satisfy condition D2. Thus, x{t) ^ x{t). 

• If there is an arrival to queue n but not m (i.e., am{t) = 0, a„(t) = 1) and x„(t) < Xjn{t), 
we conclude that x{t) and x{t) satisfy condition D3. Thus, x{t) ^ x{t). 

In all the cases we can see that x{t) di x{t). 
Note that the obtained policy n belongs to n^~^ as we applied a balancing server reallocation 
to the matching employed in tt at time slot t. 

Step 2: Construction of n for r > t: In this step, we focus on construction of oj and tt for 
T > t. We will employ mathematical induction to achieve this goal. In particular, we assume 
that u and n are constructed up to time slot r (r > t) such that x{t) ^ x{t) i.e., one of the 
conditions Dl, D2 and D3 is satisfied for x{t) and x{t). We will prove that policy n and sample 
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path Cj can be constructed such that x{t + 1) ^ x{t + 1) (i.e., one of the conditions Dl, D2 and 
D3 is satisfied for x(r + 1) and x{t + 1)). Accordingly, we consider three cases corresponding 
to each condition Dl, D2 or D3 at time slot r: 

Case 1: x(r) < x(t). In this case, the construction of Cj and vr at time slot r + 1 is 
straightforward. We let c(t+1) = c(t+1) and a(T+l) = a(T+l) and M(*)(t+1) = MW(r+l). 
Thus, x{r + 1) < x{r + 1) and therefore x{t + 1) x{r + 1). 

Case 2: x{t) is obtained from x{t) by permutation of two distinct elements m and n. In 
this case, we let c„^fc(r + 1) = Cm,k{T + 1) and Cm,k{'T + 1) = c„^fc(r + 1) for k — 1, 2, 
Ci^kir + 1) = Q,A;(r + 1) for all i e J\f, i ^ n,m and A; = 1,2,. ..,K; a„(r + 1) = am(r + 1), 
am(r + l) = a„(r + l) and ai(r + l) = ai(r + l) for i G A/", i 7^ n, m. Suppose that M*^'')(r + 1) = 
{M^I{t + 1)) Vn e TV", A; e /C be the applied matching by policy tt at time slot r + 1. We 
construct M^*) (r + 1) as follows: Let M^J (r + 1) = m/J {t + 1) for i e Af, i n,m and also 
let Mi5(r + 1) = Mi^i(r + 1) and M^^^ (r + 1) = M(;2(r + 1) for k^l,2, K. As a result, 
X (r + 1) and x{t + 1) satisfy condition D2 at time slot r + 1 and therefore x{t + 1) x{t + 1). 

Case 3: x{t) is obtained from x{t) by performing a balancing interchange of two distinct 
elements m and n as defined in condition D3. In particular, x{t) and x{t) are different in only 
two elements n and m such that x„(r) < 5;„(r) < 5m(r) < ^^(t) and the following constraints 
are satisfied: x„(t) = Xnir) + 1 and ^^(t) = a;^(T) — 1. In this case, we consider the following 
sub-cases: 

Sub-case 3.1: x„(t) < Xmir) — !: In this case, we let c(t+1) = c(t+1) and a(T+l) = a(T+l) 
and we let M(*)(r + 1) = MW(r + 1). Thus, if Xnir) = and queue n is serviced, condition 
Dl is satisfied at r + 1. Otherwise, x{t + 1) is obtained from x{t + 1) by performing a balancing 
interchange of elements m and n. Therefore x{t + 1) ^ x(t + 1). 

Sub-case 3.2: x„(r) = Xm(T) — 1: In this case again we let c(r+ 1) — c(t+ 1) and a(r+ 1) — 
a{T + 1) and we let M(*)(t + 1) = MW(t + 1). Thus, if x„(t) = and queue n is serviced, 
condition Dl is satisfied at t + 1. If queue m gets service, queue n does not get service, there is 
an arrival to queue n and no arrival to queue m, then x„(t + 1) — Xjn{r + 1) and Xm{T + 1) — 
Xn{T + 1). Therefore, x{t -\- 1) and x{t + 1) satisfy condition D2 and x{t + 1) :< x{t -\- 1). 
Otherwise, x{t + 1) and x{t + 1) satisfy condition D3 and x(t + 1) ^ x{r + 1). 

Sub-case 3.3: x„(t) = a;m(r): In this case, we let c(r + 1) = c(r + 1) and M^^'^t + 1) = 
M^'^\t + 1). Now, we consider the following cases to determine the arrivals at time slot r + 1. 
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• If Xnir) > and both queues m and n or none of them get service at time slot r + 1, 

we let a(r + 1) = a(r + 1). Therefore, if ajn{r + 1) = and a„(r + 1) = 1, x{t + 1) 
and x(t + 1) satisfy condition D2 and thus x(t + 1) ^ x(t + 1). Otherwise, x{t + 1) and 
x{t + 1) satisfy condition D3 and thus x(t + 1) ^ x(t + 1). 

• If Xjiir) > and queue n gets service at time slot r + 1 and queue m does not get service 
at time slot r + 1, we let a(T + 1) = a(T + 1). Therefore, x{t + 1) and x{r + 1) satisfy 
condition D3 and thus x{t + 1) ^ x{t + 1). 

• If Xn{r) > and queue m gets service at time slot r + 1 and queue n does not get 
service at time slot r + 1, we let a„t(T + 1) = an(T + 1) and a„(r + 1) = am{T + 1) and 
di{T + 1) = aj(r + 1) for z G A/" and i ^ m.n. Therefore, x{t + 1) and x{t + 1) satisfy 
condition D2 and thus x{t + 1) ^ x{t + 1). 

• If Xnir) — and queue n gets service at time slot r + 1 (although it does not have any 
packet to be served), we let a(T + 1) = a{T + 1). Therefore, x{t + 1) and x{t + 1) satisfy 
condition Dl and thus x{t + 1) ^ x{t + 1). 

• If Xnir) = and queue m gets service at time slot r + 1 and queue n does not get 
service at time slot r + 1, we let a„j(r + 1) = a„(r + 1) and a„(r + 1) = a„i{T + 1) and 
di{T + 1) = ttiir + 1) for z G A/" and i ^ m,n. Therefore, x{t + 1) and x{t + 1) satisfy 
condition D2 and thus x{t + 1) ^ x{t + 1). 

• If x„(t) = and neither queue m nor n gets service at time slot r + 1, we let a(T + 1) = 
a(T + 1). If a^(T + 1) = and a„(T + 1) = 1, x{t + 1) and x{t + 1) satisfy condition D2 
and thus x{t + 1) ^ x{t + 1). Otherwise, x{t + 1) and x{t + 1) satisfy condition D3 and 
thus x{t + 1) < x{t + 1). 

Note that the above cases cover all the possible cases for all of which we constructed u and tt 
such that x{t + 1) ^ x{t + 1). 

According to steps 1 and 2, from any sample path u and any arbitrary policy tt G 11^ we can 
construct a sample path u and a policy tt G n^~^ such that at all time slots we have x{t) -<p x{t). 
Therefore, f{x{t)) < f{x{t)). Consequently, the process {/(X(t))} obtained by applying policy 
TT to the system is stochastically smaller that {f{X{t))}, i.e., {f{X{t))} <st {fi^it))} and 
therefore tt G n^~^ dominates tt G H^. ■ 
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D. Proof of Lemma \^for the System with Random Service Failures 

Proof: The only difference of the proof in this case from the proof of Lemma [3] is that we 
have to consider random variables Qn,k{t) Vn G J\f,'ik G /C in our dynamic coupling argument. 
Therefore, the arbitrary sample path has the form 

a; = (x(0), c(l), g(l), a(l), x(l), c(2), g(2), a(2), x(2), ...) 
and we apply the coupling method to construct from to a new sample path 

u = (x(0), c(l), g(l), a(l), c(2), g(2), 5(2), £(2), ...) 
resulting in a new sequence of random variables 

(X(0),C'(l),g(l),i(l),X(l),C'(2),g(2),i(2),X(2),...) 

with X(0) = X(0). We will follow the same approach and consider the same cases. Before 
we proceed to the details we introduce the following complementary notation. Suppose that 
Sn \t) denotes the index of the server assigned to queue n at time slot t by policy vr. Note that 
Sn '' (t) G /C or it is empty. 

Step 1: Construction of vr for r < t: In this case, as in the proof of Lemma [3] we first 
consider r < t — 1. For those slots, we let all c(r), g(r), a(r) and M^'^^r) to be the same in 
both systems i.e., c(r) = c(r), g(r) = g(r), a(r) = a(r) and M'^'')(r) = M'^'')(r) and therefore 
we have x{t — 1) = x{t — 1). 

For time slot t, we again consider the following cases. 

1) The distance of policy tt to 11^ is less that h balancing server reallocations. In this case, 
we act the same as what we did in the proof of Lemma [3] and also let q{t) = q{t). Thus, 
x{t) ^ x{t). 

2) The distance of policy n to 11^ is exactly h balancing server reallocations. Since tt G IlJ' 
and h > 0, according to Lemma [2] there exists a balancing server reallocation such that 
either CI or C2 is satisfied. Thus, we consider two cases. 

Case 2.1: After applying the balancing reallocation, condition CI is satisfied. In other words, 
there exists a matching M^'^\t) such that if applied on the queue lengths x{t — l) = x{t — l), 
we get x'{t) such that x'{t) < x'{t). Note that if a non-empty queue is serviced under policy 
M'''"\t), it should also get service under policy tt. Otherwise, the condition x'{t) < x'{t) 
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will be violated. In this case, we let c{t) = c{t) and a{t) = a{t) and we apply M^^'^t) 
at time slot t. For any non-empty queue n that is serviced under both policies vr and tt 

we let i,4f\t)(^) = ^n,s(-)(i)(^) and = In other words, we let each 

non-empty queue n which was being serviced in both systems experience the same service 
failure. For other variables qn,kit) we let qn,k{t) = Qn,kit). Then, we can easily check that 
x(t) < x{t) and therefore x{t) ■< x{t). 

Case 2.2: After applying the balancing reallocation, condition C2 is satisfied. In other words, 
there exists a matching such that if applied on the system at time slot t, we get x'{t) which 
is different from x'{t) in two elements m and n such that x'^{t) < x'^{t) < x'j^{t) < x'j^{t) 
and the following constraints are satisfied; x'^{t) = x'^{t) + 1 and x'^{t) = — 1. We 

call such a matching by M^'^\t). Note that in this case, each queue (other than m and 
n) which is (is not) receiving service under vr is (is not) also receiving service under n. 
Queue n is receiving service under vr and queue m is not. Queue n is not receiving service 
under vf and queue m is. In this case, we let c(t) = c(t) and a(t) = a(t) and we apply 
M(*)(t) at time slot t. We let %^^(^) (^^■^{t) = q^.)^^^{t) and qi^.Wf^^^it) = for any 

queue i which is receiving service under both matchings M'^'^^t) and M^'^\t). We also let 

^m/i\t)(t) = ^n,.(-)(i)(^)' = 'lm,si:\t)(t) ^^^^ for Other failure variables we let 

Qn,kit) = qn,k{t)- By such coupling of the service success/failure random variables we will 
consider the following cases for arrivals and service failures: 

• If g = 0, we let a{t) = a{t) and therefore, x{t) = x{t) which implies x{t) ■< 
x{t). 

• If g {^)f,At) = 1, then we can use the same coupling argument on the arrival processes 
as what we did in the proof of Lemma |3] and conclude that x{t) ■< x{t). We omit the 
argument to avoid redundant discussion. 

Note that the obtained policy vf belongs to Ilf^^ as we applied a balancing server reallocation 
to the matching employed in vr at time slot t. 

Step 2: Construction of vr for r > t: In this step, the same as the proof of Lemma [3] we 
use mathematical induction to construct policy tt for r > t. Therefore, suppose that a) and vf are 
constructed up to time slot r (r > t) such that x{t) ^ x(r). Therefore, one of the conditions 
Dl, D2 and D3 is satisfied for x(r) and x(r) as follows. 

Case 1: x(r) < x{t). In this case, the construction of a) and tt at time slot r + 1 is 
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Straightforward. We let c(r + 1) = c(r + 1), g(r + 1) = q{T + 1), a(r + 1) = a(r + 1) and policy 
M(*)(r + 1) = MW(r + 1). Thus, x{t + 1) < a;(r + 1) and therefore x{t + 1) ^ x{t + 1). 

Cai'e 2: x(r) is obtained from x{t) by permutation of two distinct elements m and n. In this 
case, we couple the random variables c(r + 1), a(r + 1) and construct matching M^'^\r + 1) the 
same as what we did in case 2 of step 2 in the proof of Lemma [3l In addition to these settings, 
we let gm,fc(T" + 1) = qn,k{r + 1) and g„,fc(r + 1) = gm,fc(T" + 1) for k = 1,2, ...,K and also 
^./^(t + 1) = Q'i.fclT + 1) for all i E M, i ^ n,m and k = 1,2, K. By doing such a coupling, 
we conclude that x{t + 1) and x{t + 1) satisfy condition D2 at time slot r + 1 and therefore 
x{t + 1) ^ x{t + 1). 

Ca^e i: x(r) is obtained from x(r) by performing a balancing interchange of two distinct 
elements m and n as defined in condition D3. In particular, x(r) and x(r) are different in only 
two elements n and m such that x„(r) < x„(r) < Xml^) < ^^(t) and the following constraints 
are satisfied; x„(r) = x„(r) + 1 and Xm{T) = Xm{T) — 1. In this case, with the same argument 
as we did in the proof of Lemma [3l we can check that x(r + 1) and x(r + 1) satisfy one of the 
conditions D1-D3. We omit the details to avoid redundant discussion. 



Appendix II 

Proof of Lemma [4l Lemma [H Theorem [2] and Theorem [3] 
A. Proof of Lemma |4] 

Proof: If we write equation (fT6l) for each queue n and for r = l,2,...,t and then adding 
them up, we will have 

t K t 

Xn{t) = Xn{0) ^"''^(^) + Yl Mr)- (26) 

r=l fe=l T=l 

By multiplying the vector a to the queue length vector X(t) we have 

t t 

aX^(t) = aXT(O) - ^ aH{T)ll^ + ^ aA^(r). (27) 

r=l r=l 

Taking the expectation from both sides, dividing by t and then taking the limit as t goes to 
infinity, we will have the following. 

lim -aE[X{t)Y = lim -aE[X{0)]^ - lim - V aE[if(r)]l^ + lim - V aE[A(r)]T (28) 

r=l T=l 
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According to Lemma 3.3 in HI and the assumption that i?[X(0)] < oo, the left hand side 
term and the first term in the right hand side term are equal to zero. Using the fact that 
limt^oo J Ylt=i o:E[A{t)]'^ = aX^ the lemma follows. ■ 

B. Proof of Lemma \5\ 

Proof: Recall that M^'"\t) denotes the employed matching by policy tt at time slot t. In 
(fTTI) . by conditioning the process H{t) on the connectivity process, we will have 



= lim i fl ^E[H{r)]ll = lim ^^211 Pic)aE[H{T) \ C{t) = c]ll (29) 

For an arbitrary vector v of size |t>|, (t>)+ is defined as a vector of the same size whose i'th 
element, {v)'f , is 

if t;^ < 
Vi if Vj > 



(30) 



Therefore, (|29l ) can be bounded by 

T=i ceC 
1 * 

< lim -Y^Y. ^(^) ir?. ((")^(^ ® 
t~*oo t ^ — ^ ^ — ^ MeM 

r=l cGC 

= 5^ P(c) max ((a)+(c ® M)!^ ) . (31) 

Now, consider two different cases. 

» a E M^: The result follows directly from (|3T| ) since (a)"'' = a. 
m a ^ M^: In this case, from (|29l) we have the following inequality. 



aA^ < ^ P(c) max ((a)+(c ® M)l^) (32) 

cGC 

However, (a)+ G and according to the previous case we also have 

(«)+A^ < V P(c) max ((a)+(c ® M)l^) . (33) 

Noting (1331 ) and the fact that A G R^, we conclude that (|32l) is a redundant inequality. 
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C. Proof of Theorem |2] 

Proof: Let = {M"(c), c G C] denote a set of matchings that solve (O. A^" is not a 
unique solution of (fT9l ) and there may be more than one set of matchings A^" whose elements 
maximize Let M" = {Mf, l<i< |M°|} denote the set of all distinguished solutions of 
(fT9l) . Obviously |M"| < oo, since set M is finite. Note that each solution M'^ = {Mf (c), c G C}, 
1 < i < |M"| is corresponding to a deterministic policy with the average service rate vector 




and therefore, according to ([14)) each solution of (fT9l) is associated with a vertex of the polytope 
TZ. Since there are |M°| distinguished solution of (fT9l ). |M"| vertices of polytope 7^ are on the 
hyperplane associated to (fTSi) . In other words, each face of TZ associated to (fTSi) is represented 
by Fq, = conv .huUi<i<\M'^\Rf . 

We now introduce the following mapping. 

n-.R^^ {0,1}^ 

f if a„ = 

a = (ai,a2, •••,aAr) I — ^ /3 = /32, /^Af) : /3n = < Wn e Af (35) 

I 1 if a„ > 

Mapping i7 takes any vector a G and returns another dimensional vector /3 whose elements 
are in {0, 1}. For each element of (3 namely returns "0" if an is zero and returns "1" 

if a„ is non-zero. In the following, our objective is to prove that M" C M^*^"^ (or equivalently 
Fa C Fq(„)). To this end, we have to show that e M^(°) /or a// 1 < i < |M"|. Remember 
that Mf = {Mf (c), c G C} and Mf (c) = argmaxAfgA^ a{c ® M)l^. 

We claim that any matching M"(c) that is maximizing a{c® M)l^ also maximizes ^l{a){c® 
M)l^. Note that if a matching M"(c) is maximizing a{c® M)l^, this means that M"(c) is a 
maximum matching in a bipartite graph G" = {J\f, /C, £"') (see Figure |9l). In G", A/" and /C are 
the two sets of vertices in each part of the graph and S"' = {e";.,Vn G J\f,yk G /C} is the set 
of edges between these two parts where e";. = anCn,k- If there exists M"(c) who maximizes 
a(c® M)l^ but not ^l(a){c® M)l^ this means that matching M"(c) is a maximum matching 
for graph but not for G''^*^"^ If we treat vector ^l{a) as x'{t) in definition [3] (i.e., suppose that 
x'{t) = ^l(a) in definition [3]) and graph G^^"'^ as graph Gt, then according to Lemma |2] there 
must exist a server reallocation for matching M"(c) in graph G^^"'^ that results into another 
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matching M' whose matching weight is greater than that of M"(c). Since VL{a) G {0, 1}^ it is 
not possible for this server reallocation to satisfy condition C2. Thus, it must satisfy condition 
CI. This means that by applying the server reallocation (i.e., matching M') we will get into x'{t) 
such that x'^{t) < x'^(t) for all n = 1, 2, and there exists at least one m E {1, 2, A^} such 
that x'^{t) < x'^{t). This means that there is at least one queue m which is not receiving service 
under M"(c) but it is serviced under M' while the service of other queues is not disturbed. If 
we apply matching M' to graph we have the same results, i.e., there is at least one queue 
m which is not receiving service under M"(c) but it is serviced under M' while the service 
of other queues is not disturbed. This means that matching weight of M"(c) is less than that 
of matching M', i.e., matching M°(c) is not maximum matching in graph G". This contradicts 
with our assumption that M°(c) is maximum matching in graph G". 

Therefore, for all c G C all the maximum matchings M°(c) in graph are also maximum 
matchings in graph G'^^") . In other words, all the solutions of "arg max a;(c®M)l^" are solutions 
for "argmaxfi(a)(c® M)l^". Hence, G M^(") and therefore M" C M^("). Consequently 

Fa ^ 

Thus, according to Lemma[5]and the fact that Fa C we conclude that all the hyperplanes 

associated to half spaces (fTSi) with a G — {0, 1}^ are redundant in describing polytope TZ. 
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Note that vector a = 0^ will result in the obvious equality = and therefore is redundant too. 
Consequently, from all the vectors a G M^, justaG{0,l}^ — {0^} are sufficient to describe 
poly tope TZ and the theorem follows. 



D. Proof of Theorem |5] 

Proof: We start with the Lyapunov function evaluation, we use the quadratic function 
L{X) = J2n=i-^n(f) as our Lyapunov function. The Lyapunov drift D for two successive 
time slots has the following form. 

D{t + 1) = E[L{X{t + 1)) - L{X{t)) I X{t)] 

N 



E 



E 



n=l 



+ 2E 



For the the first term, we have 



5^X^(t + l)-X^(t) |X(t) 

n=l 
N 

5^(X„(t + l)-X„(t))2|X(t) 

1 

N 

5^X„(t)(X„(t + l)-X„(t)) |X(t) 



n=l 



E 



N 



E 



E 



^(X„(t+1)-X„(t))2 |X(t) 

N K 

Y^iMt + 1) - 1] H^At + 1))' I x{t) 

n=l k=l 

N 

5^A^(t + l)|X(t) 

n=l 

' N K 

5^^A„(t+l)if„,,(t+l) |X(t) 

;=1 k=l 

N / K \ 2 

J] + \X{t) 

n=l \k=l J 



-2E 



+ E 



N 



n=l 



K 



vfc=l 



(36) 



(37) 
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For the second term in (|36l) . we have 



E 



N 



n=l 



Y,Xn{t){X^{t+l)-Xn{t)) \X{t) 
N / K 

Y,Xn{t) L4„(t + l)-^if„^,(t+l) ) \X{t) 



E 



N 



n=l 



k=l 



N 



n=l 



n=l 



K 



X„(t)5^/7„,,(t + l) |X(t) 



k=l 



(38) 



Therefore, the Lyapunov drift D{t + 1) can be written as 



D{t + l)<NAl,, + Y,E 



n=l 



K 



N 



X^{t)Y,Hn,k{t + l)\X{t) 



k=l 



+ 2 X^{t)E[A^{t + l)]-2Y,E 

n=l n=l 

In the following, we show that the Lyapunov drift in (|391 ) is bounded as follows. 

^ 2 

D{t + l)<NAl,^ + Y,E 



(39) 



n=l 



' K \ ^ 

Ycn,k{t + l)Mn,k{t + l)\ \X{t) 



TV 



N 



+ 2 ^ X„(t)E[A„(t + 1)] - 2 ^ E 



X„(t) ^ C„,fc(t + l)Mn,k{t + 1) I X{t) 



_ (40) 

n=l ?i=l L fc=l 

where c(t + 1) = (c„ + 1)), n G A/", /c G /C is the observed connectivity matrix at the beginning 
of time slot t + 1. To prove (|40|) . note that for each queue n E N one of the following conditions 
is satisfied. 

• Ylk=i Hn,k{t + 1) = Z]f=i Cn,k{t + l)Mn,k{t + 1): In this case, we can easily check that 

E[Xl{t + l)-Xl{t)\X{t)]<NAl 



max 



+E 



^c„,fc(t + l)M„,fc(t + l) \X{t) 

\k=l J 



+2X„(t)EK(t+l)]-2E 



K 



Xn{t) Cn,k{t + l)M„,fc(t + 1) I X{t) 



k=l 



(41) 



• Ef=i HnAt + 1) < Ef=i Cn,fe(t + l)M„,fc(t + 1): In this case, J2k=l ^nA^ + 1) = ^n(t) 
and +1) = An{t + l) as there are not enough packets in queue n to be served at time 
slot t + 1. It is not hard to check that inequality (|4T)) is also satisfied in this case. 
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According to the above discussion, we can observe that inequality (|4TI) is satisfied for all the 
queues and therefore (|40l) follows. 

Using the the fact that Yl!k=i '^n,k{t + 1)M„ ^(t + 1) > 0, we have the following inequality. 

N / K \'^/NK \2 

I] + + < I]I]c„,fc(t+l)M„,,(t + l) <{min{N,K}f{A2) 

n=l \k=l / \n=l k=l / 

Hence, the Lyapunov drift (|40l) is bounded by 

N 

D{t + 1) < iVAL. + {min{N, K}f + 2 X„(t)i?[A„(t + 1)] 



n=l 



TV 



it 



-2^E X„(t)5^c„,fc(t + l)M„,fc(t + l) I X(t) . (43) 

n=l L fc=l 

By conditioning the last term of (l43l) on the connectivity process at time slot t + 1, we will have 

D{t + l)<NAl^, + {min{N,K}f 



+2E 



(44) 



Note that matching M in (|44l) depends on the selected policy. According to ([3]), we can see 
that by selecting MWM policy, the second term of (l44l) will be minimized and therefore the 
right hand side term in (|44|) will be minimized over all the existing server assignment policies. 
In other words, for MWM policy and any arbitrary server assignment policy vr we have 

+ 1) < iVAL. + (mm{iV, K})' 



+ 2E 



X{t) A^{t + 1) - 5^ P(c) (c ® M(™)(t + 1)) ll I X{t) 



cec 



<NAl^^ + irmn{N,K}f 



+ 2E 



X{t) A'it + 1) - J] P(c) (c ® mW {t + 1)) 1^ I X{t) 



cec 



(45) 



where D^^{t + 1) is the Lyapunov drift of MWM policy at time slot t + l. 

It is important to note that the set of inequalities in (|2T]) determines an open polytope IZ' for 
which we have IZ' = IZ — hound(lZ). In fact. Theorem [3] states that if the vector A is strictly 
inside polytope IZ then the system is stable. Now, consider the system of Figure [T] with arrival 
processes for which we have A G 7^' at each time slot t. Suppose that 5 is a positive real number 
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such that \ + Slj^ G houndilZ). According to our discussion about deterministic and randomized 
policies, there exists a randomized policy M^""^ = {M''"'^(t)}^^ for which we will have 



E 



,cec 



Therefore, 



E 



N 



x{t) l^A'it + ^(c) ® + 1)) llj I x{t) 

Putting together from (|44l) to (|47] ), we conclude that 

AT 

D™(t + 1) < iVAL. + (mm{iV,ir})2 - 26j2Xn{t), 



n=l 



n=l 



(46) 



5^X„(t). (47) 



(48) 



and according to Lemma 4.1 in [4J, this proves the stability of MQMS system. Furthermore, we 
can conclude the upper bound given in (|22)) for the average total queue occupancy of MWM 
policy. ■ 
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